Hair curl measurement

ABSTRACT

A method of measuring indications of hair type of a user, the method comprising the steps of: providing a mobile device; providing a reference card, the reference card including one or more reference markers; providing one or more of the user&#39;s hairs at the 5 surface of the reference card; acquiring one or more images of said one or more hairs using the mobile device; identifying the path of each of the one or more hairs along the reference card; and calculating the curve of each hair from its identified path.

FIELD OF THE INVENTION

The present invention relates to a system and method for measuringindications of hair type of a user, more particularly to a systemcomprising a mobile device and a reference card.

BACKGROUND OF THE INVENTION

Consumers of hair products are able to select from a high volume ofavailable products, due in part to the fact that many hair care brandshave increasingly large product ranges.

These product ranges may cover various product categories including:shampoos, conditioners, treatments and styling products. Within eachproduct category, a brand may produce multiple versions of a productwith the ingredients of each version chosen to address the particularneeds of a given target hair type. For example, a given hair type mayinclude one or more of the following: straight hair, wavy hair, curlyhair, thick hair, frizzy hair, thin hair, colour treated hair, anddamaged hair.

Each version of a product could be clearly marketed for its target hairtype. However, it may be difficult for a consumer to accurately identifywhat hair type they actually belong to and the vast array of choicecould therefore lead to confusion. When a consumer chooses a version ofa product, their choice is usually based on nothing more than their ownperception of their hair type and this perception may be inaccurateparticularly if it is based on external factors such as advertising orrelative comparisons with friends and family. In addition, terms such as“thick hair” or “thin hair” can be open to misinterpretation. Forexample, a consumer may have many hair strands, but each hair strand maybe of a thinner than average diameter.

The optimal choice for hair products could depend on how a consumerstyles their hair. For example, a consumer may have naturally curly hairbut may blow dry their hair straight. In such a scenario, the consumermay find it difficult to categorise their own post-styling hair type.They could consider it to be curly, straight, or somewhere in-between.

In the current market, it is therefore easy for consumers to wronglyassign themselves a specific hair type and to select a specific versionof a product, the ingredients of which may not be the best match for theactual characteristics of their hair. Such selection of a less thanoptimal product version can lead to poor customer satisfaction.

There is therefore a need for consumers of hair products to be able tomake more accurate, more informed choices about what version of ahaircare product is best for them.

SUMMARY OF THE INVENTION

While it is possible to gain a macroscopic appreciation ofcharacteristics such as hair thickness and curvature using the naked eyeor conventional optical microscopes, such examination is invariablyunreliable due, for example, to variations in the eyesight of differentusers. So, choosing a haircare product based on such a macroscopicexamination is unlikely to lead to the optimal choice of haircareproduct, and resultant customer satisfaction. At its broadest, thepresent invention addresses this problem by providing a more reliable,and easily-usable method/system for measuring indications of the hairtype of a user, which do not rely on the user's discernment. Focusedhaircare recommendations can therefore be automatically provided, in amanner which requires minimal human input, and the scheme of provisionfor which does not vary from user to user.

The present invention achieves this by providing, according to a firstaspect, a method of measuring indications of hair type of a user, themethod comprising the steps of: providing a mobile device; providing areference card, the reference card including one or more referencemarkers; providing one or more of the user's hairs at the surface of thereference card; acquiring one or more images of said one or more hairsusing the mobile device; identifying the path of each of the one or morehairs along the reference card; and calculating the curve of each hairfrom its identified path.

In this way, the first aspect of the invention may provide a method ofmeasuring indications of a hair type of a user using a mobile device,and a reference card including one or more reference markers, the methodincluding the steps of:

-   -   providing one or more of a user's hairs on the surface of the        reference card;    -   acquiring one or more images of said one or more user's hairs        using the mobile device;    -   identifying the path of each of the one or more hairs along the        reference card; and    -   calculating the curve of each hair from its identified path.

In the present application, the term “hair type” may be understoodamongst other things to refer to the thickness or diameter of the hair,or the curvature, curve or curl of the hair (such terms may be usedinterchangeably).

Accordingly, the thickness or diameter of the hair may also becalculated from the acquired image, in addition to the calculation ofthe curve. By calculating these from acquired images, better results maybe achieved than the very coarse estimates of thickness and curvature ofa hair which a user may be able to provide themselves without employingthe method of the first aspect of the present invention, e.g. just byexamining the hair by eye.

The step of providing one or more of a user's hairs on the surface ofthe reference card may include simply placing the card on the surface ofthe reference card. In methods according to the present invention, asdiscussed later, there is no requirement for securing the hair flat onthe surface, which may distort calculations relating to both diameterand curve of a given hair.

The image acquired may include a plurality of pixels, each having anassociated light intensity value. The image includes the geometricprofile of each hair on the surface of the reference card, whichrepresents the shape of that hair within the image, and may be detectedusing the light intensities of the pixels within the image. In otherwords, each hair has an associated geometric profile within the acquiredimage. The geometric profile of the hair may be considered to includepath information (or “the path”) and dimensional information (or“dimensions”). Dimensional information preferably includes the diameteror thickness of the hair in question. The path represents the directionof a given hair at all points along its length, or in other words theroute which a given hair takes through the image. Path information anddimensional information are preferably separable.

Accordingly, the diameter of the hair may be calculated from thegeometric profile of the hair. The identified path is also useful in thecalculation of the diameter of the hair. Consider, for example, a curlyhair, which crosses over itself in the acquired image, i.e. where thepath of the hair crosses itself. Here, identification of the path of thehair ensures that it is known that there is only one hair which crossesitself, rather than several hairs crossing each other. Thus, theidentification of the path of the hair ensures that superfluous diametercalculations are not performed. A similar situation occurs when thereare more than one hairs present on the reference card. Here,identification of the path of each hair ensures that diametercalculations are performed for each hair, so that neither too few nortoo many calculations are performed. In the two preceding examples, oncethe path has been identified the calculation of the diameter may beperformed from the geometric profile of each respective hair.

As will be appreciated from the above, no alignment of the one or morehairs is necessary, and when a plurality of hairs are used, it does notmatter if there is any overlap or crossing of hairs. Nor does it matterif a single hair crosses over itself, due to the path identificationstep of the present invention.

In this way, the claimed system provides a simple yet accurate mechanismfor quantitative analysis of the hair. By taking actual measurements,the consumer is more likely to correctly identify their hair type. As aresult, based on this identification, they are more likely to choose theoptimal product for their hair as errors which arise from qualitativedescriptions of hair types can be prevented. The likelihood of customersatisfaction is therefore increased.

Advantageously, the system can be implemented by the user themselves athome and does not require any complicated or expensive laboratoryequipment.

The reference markers on the reference card allow for calibration of theimage, particularly with regard to the dimensions of the hair.

For example, the reference markers may indicate a known dimension whichmay then be used in subsequent image processing, particularly in thecalculation of the diameter of the hair. Specifically, calculation ofthe diameter of the hair may include a comparison of the diameter of thehair in the acquired image (e.g. from the geometric profile), with aknown dimension indicated by the reference marker(s). Accordinglyinformation regarding the reference marker is preferably used in thecalculation of the diameter, or other geometric properties. Otherbeneficial features of the reference markers are set out later in theapplication, with reference to other structural and functional featuresof the invention.

Optional features of the invention will now be set out. These areapplicable singly or in any combination with any aspect of theinvention.

Advantageously, there is no need for the reference card to include anykind of attachment portions or adhesive region(s). This is particularlyadvantageous for any measurements of curly hair because curly hair hasan inherent resilience and it may therefore more difficult to attach tothe card.

The step of acquiring the image may involve a refocusing step if imageappears to be out of focus on the device.

Acquiring the one or more images may be performed simply by holding themobile device above the reference card, and actuating an image capturefunction perhaps using a dedicated button or a touchscreen element ofthe mobile device.

The mobile device may comprise a digital imaging device (e.g. a camera)for capturing the image of the one or more hairs on the reference card.In this way, the mobile device is configured to capture the image aswell as to acquire and process it. In addition to the digital imagingdevice, the mobile device may include a display.

The display may be configured to display acquired images captured usingthe digital imaging device. The mobile device may include a databasestoring a library of images, and preferably the mobile device may beconfigured to store images acquired using the digital image capturingdevice in the database. The mobile device may be configured to retrieveimages from the database for further processing, where said furtherprocessing may include any method steps disclosed herein which areperformed subsequently to the step of acquiring one or more images ofthe one or more hairs. Optionally, the step of acquiring the one or moreimages includes the steps of: providing a real time view (e.g. on thedisplay) of the input of a digital imaging device on the mobile device;providing an overlay on top of the real time view, the overlay havingthe same shape as one or more of the one or more reference markers.

When the overlay has the same shape as the reference markers, thisensures that the user is able to position the mobile device such thatthe overlay overlaps the reference marks shown in the real time view.Consequently, a desired alignment may be achieved as the image isacquired. By lining up the overlay with the reference marks in the realtime view, both a desired orientation and a desired distance from thereference card can be achieved. Clearly, this is advantageous since suchan orientation/distance can be selected which provides a desirableresolution for the acquired image, or which allows a wider field ofview, (i.e. making sure that the one or more hairs are shown fully inthe acquired image). Accordingly, in some embodiments of the method, theentirety of each of the one or more of the user's hairs is providedwithin one or more of the reference marks, i.e. no part of any hair liesoutside one or more of the reference markers.

Optionally a first overlay is provided for alignment with a firstreference marker during a first image capture step and a second overlayis provided for alignment with a second reference marker during a secondimage capture step.

Optionally, the step of acquiring an image of said one or more of theuser's hairs includes: obtaining a first image of the one or more hairson the reference card, the first image being taken at a first distancefrom the reference card; and obtaining a second image of the same one ormore hairs on the reference card, the second image being taken at asecond distance from the reference card; wherein the second distance isless than the first distance.

By taking two images at different distances, different levels of zoomare achieved. A more zoomed-in image, i.e. the second image provides acloser view of the one or more hairs and accordingly may provide ahigher resolution which may be advantageous for e.g. identification ofedges within the image, or calculation of diameter from the image (orthe geometric profile). Similarly, a close-up image from the seconddistance may also allow more accurate determination of the path of theone or more hairs shown within the second image.

In contrast, the first image from the (larger) first distance allows awider field of view. In this way, the first image will contain a greaterproportion, and preferably all, of each of the one or more of the user'shairs. This enables the path to be identified over a greater length ofeach respective hair. By maximizing the extent of the hair which is inthe image, it is possible to ensure with more certainty that the correctnumber of hairs is identified (taking into account e.g. hairs crossingover themselves and over other hairs). This is also particularly usefulin embodiments in which the curve of the hair is also calculated, sincethere is more data (i.e. data over a greater length) available for eachrespective hair.

In some embodiments, the first reference marker is for use with thefirst image (at the first distance), and the second reference marker isfor use with the second image (at the second distance). The referencemarkers in this case may be associated with the first and secondoverlays, as discussed earlier. The presence of first and secondreference markers ensure an appropriate alignment of the first image,relative to the second image. Again, here “alignment” refers to acombination of both the distance and the orientation. Specifically, itensures an appropriate alignment of position of the mobile device whenacquiring the first image, relative to the position of the mobile devicewhen acquiring the second image. The size of the first reference markermay be selected so that, when it is aligned with the first overlay, themobile device is at the first distance. Similarly, the size of thesecond reference marker may be selected so that, when it is aligned withthe second overlay, the mobile device is at the second distance. Thefirst and second distance may be predetermined, or user-set.

Similarly, the shapes of the first and second reference markers may beselected so that when they are aligned with the respective overlay, themobile device is at the same orientation with respect to the referencecard. This ensures that the images are acquired from the same direction,and therefore that the first and second image show a given hair or hairswith the same projection. Consequently, and importantly, this means thatthe geometric profile of the one or more hairs in the first and secondimage is the same, viewed only at a different level of zoom. This meansthat the path is substantially unchanged, and ideally completelyunchanged, the second image relative to the first image. This isparticularly important, since the one or more hairs need not be securedto the reference card, and accordingly the projection from slightlydifferent angles may be substantially different.

As discussed, a given hair type may include one or more of thefollowing: straight hair, wavy hair, curly hair, thick hair, frizzyhair, thin hair, colour treated hair, and damaged hair. To provide animproved accuracy when identifying hair type, it may be desirable tomeasure more than one indication of the type of hair. For example, themethod of the present invention may measure one or more characteristicsof the curl of a user's hair as well as other indications such as thediameter of their hair. Knowledge of multiple indications of hair typeis particularly useful when choosing suitable hair products. Forexample, products best suited for straight thick hair may not be optimalfor thin curly hair, thick curly hair, or thin straight hair.

Curl algorithms for calculating the curl of a hair from an image maygive the best results when more of the length of the curl lies withinthe image. Other algorithms such as those used to calculate diameter,may give the best results when the numbers of pixels per hair diameterare maximised. Thus, where measurements of multiple indications of thehair type are desired, overall results can be improved by obtaining twoseparate images; one taken over a larger capture area and one close up.The close up that forms the second image is preferably taken of a zoomedin region within that capture area.

Optionally, the reference card comprises: a first reference marker foraligning the mobile decide with the reference card when obtaining thefirst image; and a second reference marker for aligning the mobiledevice with the reference card when obtaining the second image.

The first reference marker may take the form of an outer marker forexample a rectangle or other polygon. One advantage of a rectangularmarker is that it can be formed of the same relative dimensions as thescreen of a mobile device which optimises the total pixels available onthe mobile device.

The first reference marker may be a solid line around the perimeter orseparated from the perimeter of the reference card to form a boarder.

The reference card could be provided as part of marketing material, forexample on an advert or article in a magazine.

Optionally, a plurality of the user's hairs are provided at the surfaceof the reference card.

A single hair can be measured. However, measuring multiple hairs maygive more consistent readings due to variation between hairs from aperson's head

Optionally, at least three of the user's hairs are provided at thesurface of the reference card.

Optionally, the image(s) obtained of the hairs is a single imagecomprising all three hairs. Where a first and a second image are taken,the hairs are not moved in-between the two images.

Identifying the path of a given hair refers to the identification of thepath in the geometric profile of that hair in the image acquired by themobile device. Identification of the geometric profile may include aninitial step of detecting the location of the hair in the acquiredimage. Such a detection may be based on the light intensity value ofpixels within the acquired image. When the location of the hair isdetected, the path which it forms may be parameterized, for exampleusing a polynomial fit. However, there are numerous other examples ofmethods by which the path of the hair may be identified.

For example, in other embodiments, the step of identifying the path ofeach of the one or more hairs along the reference card may comprise:applying an edge detection algorithm to the image; identifying thecentre line of the one or more hairs at a plurality of points along thehairs; and applying a fit to the plurality of points.

Optionally, the step of applying a fit to the plurality of pointsincludes the step of applying a polynomial fit.

Optionally, the method of measuring indications of hair type of a userfurther comprises the step of extracting the diameter of the one or morehairs from the one or more acquired images.

According to a second aspect of the present invention, there is provideda system for measuring the hair type of a user, the system comprising: amobile device (20); and a reference card (10), the reference cardincluding one or more reference markers; wherein the mobile device isconfigured to: acquire one or more images, each of the one or moreimages being an image of one or more hairs of the user; identify thepath of each of the one or more hairs along the reference card; andcalculate the curve of each hair from its identified path.

The configuration of the mobile device to convert the image(s) into hairmeasurements may take the form of a computer program product, tangiblyembodied in a non-transitory computer readable medium, the computerprogram product including instructions for carrying out the method stepsof the first aspect required to turn the image taken by a digitalimaging device of the mobile device into the hair indicationmeasurements such as curl and/or diameter.

For example, the computer program may be an app which is downloadableonto a mobile device.

The term “mobile device” may be understood to mean “portable electronicdevice”, specifically a portable electronic device configured to receivea visual input. Accordingly, the mobile device could be any electronicdevice such as: mobile phone/cellphone, tablet, phablet, laptop or adigital camera. As was discussed earlier in the application, the mobiledevice may comprise a digital imaging device (e.g. a camera) forcapturing the image of the human hair attached to the reference card. Inthis way, the mobile device is configured to capture the image as wellas to process it.

The surface of the reference card may be coloured white to make thecolour contrast as great as possible between the hair to be measured andthe card surface. A white surface would be particularly optimised fordark hairs (e.g. brown, black). For blond or grey hairs it may bepreferable to include a dark surface to invert the colour contrast. Asingle reference card may be configured to be used with multiple haircolours by including a surface with a while background and a surfacewith a black background. White and black are used as examples, but couldbe substituted by other light or dark colours.

Optionally the reference card may contain instructions on its use or onthe applications use. It may also contain a website link and/orinstructions for downloading the computer program (application) forcarrying out the claimed method.

Optionally the reference card has a matt surface. This surface may havea roughness more than or equal to the roughness of matt printer paper.This may correspond to a TAPPI 75 gloss value below 35%, even morepreferably this may correspond to a TAPPI 75 gloss value of no more than10%.

In this way, the amount of light absorbed by the card is increased.Reflections of light from the card are therefore minimised so that lightspots in the image due to reflections of light sources are reduced. Thisis particularly important when the image is taken in artificial light.Bright spots in the image can affect the processing of the image so amatt surface of the reference card gives rise to a robust system thatcan be used in various lighting conditions or changing lightingconditions.

Where images are recorded in well-lit environments, it is particularlydesirable for any flash on the mobile device to be disabled before animage is recorded. Where a flash might be useful (for example in anextremely poorly lit environment) the matt surface will help to minimisethe effect of bright spots which arise as a result of the reflection ofthe flash.

Optionally, the reference card comprises: a first reference marker foraligning the mobile decide with the reference card when obtaining afirst image using the mobile device; and a second reference marker foraligning the mobile device with the reference card when obtaining thesecond image using the mobile device.

In some embodiments the size and/or shape of the reference card itselfmay be used as a reference marker instead of or in addition to acalibration marker. The measurements of the reference card could includethe height, length or any other measurement of a part the reference cardsuch as distance between diagonally opposing corners.

Optionally, the second reference marker is located within the firstreference marker. This may, for example, ensure that the second image istaken at a smaller distance than the first image as is discussedearlier.

Optionally, the mobile device is configured to present the image takenby the digital imaging device to the user for quality verificationbefore processing the image.

According to a third aspect of the present invention there is provided areference card for use with a computer program to measure indications ofhair type of a user, the reference card including: a first referencemarker for aligning the mobile decide with the reference card whenobtaining a first image using the mobile device; and a second referencemarker for aligning the mobile device with the reference card whenobtaining the second image using the mobile device; wherein the secondreference marker is located within the first reference marker. Such areference card may be used in the method of the first aspect of theinvention.

According to a fourth aspect of the present invention, there is provideda computer program for measuring indications of hair type of a user, thecomputer program configured to carry out the steps of: acquiring one ormore images of one or more hairs on a reference card using a mobiledevice; identifying the path of each of the one or more hairs along thereference card; and calculating the curve of each hair from itsidentified path.

Optionally, the step of acquiring the one or more images includes thesteps of: providing a real time view of the input of a digital imagingdevice on the mobile device; providing an overlay on top of the realtime view, the overlay having the same shape as one or more of the oneor more reference markers.

In this way an alignment aid is provided to ensure that the imagesacquired are taken at a suitable distance and angle such that theresolution will be sufficient and no parallax error will be introduced.

Where compatible, optional features set out above with respect to thefirst aspect of the invention may also be included in any or all of thesecond, third and fourth aspects of the invention, and vice versa.Further optional features of the invention are set out below.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described by way of examplewith reference to the accompanying drawings in which:

FIG. 1 shows a schematic diagram of a reference card;

FIG. 2 shows a schematic diagram of a mobile device;

FIG. 3 shows a schematic diagram of a mobile device and reference cardin communication with a network;

FIG. 4 shows a process for measuring the hair type of a user;

FIG. 5 shows a process for extracting measurements from images;

FIG. 6 shows an example of image correction steps carried out as part ofthe process for extracting measurements from images;

FIG. 7 shows an example of hair path identification steps carried out aspart of the process for extracting measurements from images;

FIG. 8 shows an example of curl measurement steps carried out as part ofthe process for extracting measurements from images;

FIG. 9 shows an example of diameter measurement steps carried out aspart of the process for extracting measurements from images;

FIG. 10 shows an example of providing hairs to be measured at thereference card of FIG. 1;

FIG. 11 shows an instruction display on the mobile device;

FIG. 12 shows an example of obtaining a first image of the hairs andreference card of FIGS. 10 and 11;

FIG. 13 shows an example of obtaining a second image of the hairs andthe reference card of FIGS. 10 to 12;

FIG. 14 shows an example of a further instruction screen;

FIG. 15 shows an example of one way of transferring data from the mobiledevice to a computer module in an external computer;

FIG. 16 shows an example of an output of extracted measurements;

FIG. 17 shows an example of shows an example of extracted results storedby an external computer;

FIG. 18 shows an example of switches of hair grouped according to curl;

FIG. 19 shows three hairs from each group of FIG. 18 placed on thereference card of FIG. 1;

FIG. 20 provides an overview of hair curl descriptions;

FIG. 21 depicts examples of measurements of some characteristics ofcurl;

FIG. 22 depicts alternative examples of measurements of somecharacteristics of curl;

FIG. 23 depicts the examples of measurements of some characteristics ofcurl as shown in FIG. 22, but for hair with a different curl;

FIG. 24 shows curl measurement results for each of the hair types shownin FIGS. 18 and 19;

FIG. 25 shows the curl measurements of FIG. 24 with curl categoriesapplied;

FIG. 26 shows an example of diameter measurement results; and

FIG. 27 gives an example of diameter ranges for characterisation.

DETAILED DESCRIPTION AND FURTHER OPTIONAL FEATURES OF THE INVENTION

A system for measuring the indications of hair type of a user isdescribed with reference to FIGS. 1 to 3 below.

The system comprises a mobile device 20 and a reference card 10, thereference card comprising a first reference marker 12 a and a secondreference marker 12 b, 12 c, 12 d, 12 e on the card.

The first reference marker 12 a forms an outer reference marker. In theembodiment shown in FIG. 1, this is a rectangle. The relative dimensions(i.e. the ratio between the length and the width) of the rectangle arethe same as the relative dimensions of the perimeter of the referencecard. They are also the same as the relative dimensions of the screen onthe mobile device (i.e. of the array of pixels of the digital imagedevice).

The second reference marker is formed from a plurality of separatereference markers arranges in an array. In this case, the secondreference marker comprises four separate filled in squares, all four ofthe squares arranged in a rectangular array, the array lying within thefirst reference marker.

The outer points of the array of the plurality of separate referencemarkers would, if joined up, form corners of a rectangle having the samerelative dimensions as that of the first reference marker.

The reference card 10 includes a surface 13 against which the hairshould be located before measurement takes place. In the embodiment ofFIGS. 1 to 3, the surface is a light colour such as white in order tomaximise contrast between it and dark hairs placed upon it (e.g. brownor black hairs).

FIG. 2 shows an illustrative embodiment of a mobile device which issuitable for practicing the various aspects and embodiments of thepresent invention. The mobile device may include all of the componentsshown but may contain more, or less.

The mobile device 20 includes a digital imaging device 60 such as adigital camera for recording digital photographs. These photographs maythen be stored in a data storage section of the memory 22.

The mobile device 20 shown includes a central processing unit (CPU) 21in communication with a memory 22 and various other components.

These other components of the mobile device 20 shown include a powersupply 23, a network interface 24, a display 25, an input/outputinterface 26, an audio interface 27, a flash 28 and user controls 29.

The power supply 23 provides the power used by the mobile interface andmay take the form of a rechargeable battery and/or external powersource.

The network interface 24 provides a mechanism for the mobile device tocommunicate directly or indirectly with any other computing device andincludes circuitry configured for use with one or more communicationprotocols and technologies including but not limited to: GPRS; GSM;TDMA; transmission control protocol/Internet protocol (TCP/IP); CDMA;WCDMA; Wi-Fi; 3G, 4G, Bluetooth or any other wireless communicationprotocols.

The display 25 may be an LCD (Liquid crystal display), a plasma displayor any other suitable electronic display and may be touch sensitive inthat it may include a screen configured to receive an input from a humandigit or a stylus.

Input/output interface(s) 26 may include one or more ports foroutputting information e.g. audio information via headphones, but mayalso be an input port configured to receive signals including remotecontrol signals.

The audio interface 27 typically includes a speaker which enables themobile device to output signals and a microphone which enables themobile device to receive audio signals including voice control inputsfor use in controlling applications.

The mobile device 20 shown includes a flash 28 which may be used inconjunction with the digital imaging device to illuminate an object ofwhich a photograph is being taken.

User controls 29 may take the form of external buttons or slider whichallow a user to control various functions of the mobile device.

An application saved on the device may be configured to interact withthe various components of the device such that upon receiving an inputfrom one or more of the user controls, the digital imaging device and istriggered and a digital photograph is taken of an object (such as thehair on the reference card). This image may then be stored in the memoryand one or more algorithms may be used to process the stored image.

The computer program described herein may take the form of anapplication stored in the memory 22.

The mobile device may be connected to an external computer 30 eitherdirectly or via a network 40 so that computationally extensivecalculations can be carried out by a computational module on theexternal computer, the external computer being more powerful than themobile device and therefore capable of performing the calculations morequickly.

The mobile device 20 may also be configured to exchange information withother computers via a network 40. The network may include the internetand/or one or more local area networks (LANs) or wide area networks(WANs).

As shown in FIG. 3, this exchange of information may include accessing aproduct database 33 on an external computer 30. The product database mayinclude a list of suitable products and an indication of the diameter ofhair for which the product is most suitable. This indication could be anupper diameter threshold, a lower diameter threshold, or a range ofsuitable diameters.

In this way, processed information extracted from an image taken of auser's hair on the reference card 10 can be compared and matched againstthe indication in the product database. Selection criteria can then beapplied to select the most suitable product for the user based on thenumerical value or label.

In other embodiments, the product database may be stored within thememory 22 of the mobile device.

It should be understood that information such as the measurements takenand/or product selection can be communicated with external servers viathe network 40. For example, the user could share this information withothers by posting the information on social media websites.

A method of measuring indications of hair type of a user is describedbelow with reference to FIGS. 4 to 17, where like reference numeralscorrespond to features described in relation to FIGS. 1 to 3.

A reference card 10 is provided along with a mobile device 20. The hairto be measured is placed s2 onto the reference card 10 so that the hairlies against the surface 13 of the reference card. The direction of theplacement of the hair is not important. Hairs can simply be dropped ontothe reference card. This step is shown in more detail in FIG. 10.

A first image of the hair on the reference card is taken (s3) using thedigital imaging device 60 of the mobile device. The first referencemarker 12 a is used to align the image. This ensures that the image istaken at a first distance.

A second image of the hair on the reference card is taken (s4) using thesame digital imaging device 60 of the same mobile device. The secondreference marker in the form of the plurality of separate markers 12 b,12 c, 12 d, 12 e is used to align the image. This ensures that the imageis taken at a second distance which is less than the first distance.

The image(s) may be stored in the memory 22 before being processed.

Once the images have been acquired, measurements are extracted s5 fromthem.

As discussed in relation to FIGS. 5 to 9 below, the extraction of themeasurements includes the steps of identifying the path of each of theone or more hairs along the reference card; and calculating the curve,diameter and other indications of each hair from its identified path.

FIG. 5 shows an overview of a process (s5) for extracting measurementsfrom images. The process in the example shown comprises four mainstages:

-   -   Image correction (s51);    -   Hair path identification (s52);    -   Curl measurement (s53); and    -   Diameter measurement (s54).

The portions of the algorithm which correct the image and detect thepath of the hair are shared, the calculation of curve and diameter fromthe path are separate, as described in more detail below.

FIG. 6 shows an example of the image correction stage (s51).

Firstly, an adaptive threshold is applied to the image (s51 a).

Contours are then found in this threshold and the hierarchy of thesecontours is calculated (s51 b).

Contours matching those expected for the first or the second referencemarker are found (s51 c).

In the example shown, the four found corners of the marker (either thefirst reference marker or the second reference marker), and the knownsize of the marker are used to calculate a homography; a mapping betweenthe distorted image and the true size of the marker (s51 d). The imageis then remapped to appear with the marker undistorted with bicubicinterpolation.

FIG. 7 shows an example of hair path identification stage (s53).

Firstly, the image is converted to grayscale (s52 a) (for example bytaking only the green channel from the optical input of the digitalimaging device).

Edge detection is then performed (s52 b. This could be achieved byapplying Sobel filters.

The centre line of the hair is identified (s52 c) as being the centrepoint of two strong parallel edges facing opposite orientations within athreshold distance.

Morphological operations such as “bridging” “closing” and “thinning” arethen preformed on the centre points (sometimes referred to as candidatepoints) to improve the data and remove noise (s52 d).

Connected components are used to join edges (s52 e) which are touching(in a Moore neighbourhood sense) and are not at a junction (i.e. acrossover between two or more hairs).

Lines are fit (s52 f) to provide a curve for each hair. This includesfitting to the pixels of the centre points to produce many “curve” eachcomprised of many straight line “segments”. All the curves are comparedto each other and are merged if their ends lie close together and are insimilar orientation. Very short curves below a given length thresholdare removed to avoid anomalous results. This produces a list of curveseach composed of segments identified by the x and y co-ordinates oftheir ends. In an ideal case a single curve will correspond to a singlehair path for a single hair.

FIG. 8 shows an example of curl measurement stage.

Firstly, for each curve the curvature is calculated at many points alongits length (s53 a), for each calculation:

a. The adjacent points in a window are averaged using a Gaussian kernel

b. A polynomial is fit (s53 b) to the resulting points

c. The curvature of the polynomial function is calculated

Secondly, the algorithm returns the median curvature over all curves(s53 c)

Thirdly, the algorithm calculates the “distance index”, for each curve(s53 d):

a. The mean point of the curve is calculated

b. The distance of each segment from this mean point is calculated

c. The median of all these distances is calculated

d. The distance index is calculated as the ratio of the mediancalculated in c above to the arc length of the curve

e. This number is then weighted by the length of the curve.

FIG. 9 shows an example of diameter measurement stage (s54). As with thecurl measurement, this uses the data from the identified hair paths sothese must first be obtained (s54 a).

For each curve, the diameter is calculated at many points along it, thisis done as follows:

The orientation of the hair is measured (s54 b) from the hair path(s),and a cross section taken perpendicular to this (s54 c).

Bicubic interpolation, or another suitable method is used to take anintensity profile (s54 d).

The intensity profile is used to locate the edges of the hair (i.e. thehair width) (s54 d). Two peaks are found in the intensity profile andthe maximum of these are taken to be the diameter (s54 e).

The median is taken of all these readings to give the average hairdiameter. This could be the median over a plurality measurements on onehair, or over a plurality of measurements over more than one hair.

Unlike other known types of diameter measurement, there is therefore noneed to have aligned or oriented the hair in a specific way relative tothe hair card. This is particularly advantageous for diametermeasurements of curly or unruly hairs which can be difficult to align.However it also provides an efficient measurement technique for hair ofall types.

User operation in order to carry out the method can be understood withreference to FIGS. 10 to 17 where like reference numerals are used forthe features described in FIGS. 1 to 3.

FIG. 10 shows an example of providing hairs to be measured at thereference card of FIG. 1 by placing the plurality of hairs onto thereference card.

FIG. 11 shows an instruction display on the mobile device 20; theinstruction display providing the user with instructions of the stepsrequired to carry out the step of acquiring the first image. Theinstruction page contains a link that the user can click on to proceedto the image capture when they are ready.

The capture of the first image at a first distance can be seen in FIG.12. The screen of the mobile device exhibits the optical input receivedby the digital imaging device (i.e. the camera) of the mobile device. Ascan be seen from FIG. 12, the computer program (application) running onthe mobile device provides an overlay 112 on top of the live feed;

the overlay having the same relative dimensions as the first referencemarker. The user aligns the inner edge of the overlay boarder with theouter edge of the first reference marker as viewed through the screenbefore recording the image by pressing a capture button on the screen.

FIG. 13 shows the capture of the second image at a second distance. Thescreen of the mobile device exhibits the optical input received by thedigital imaging device (i.e. the camera) of the mobile device. As can beseen from FIG. 13, the computer program (application) running on themobile device provides a second overlay 112 a on top of the live feed;the second overlay having the same relative shape and dimensions as thesecond reference marker. The user aligns the inner edge of the overlayboarder with the outer edge of the first reference marker as viewedthrough the screen before recording the image by pressing a capturebutton on the screen.

FIG. 14 shows a further user instruction screen; this time confirmingthat the images have been taken, displaying the images, and instructingthe user to connect the mobile device to an external computer foranalysis. A direct connection is shown as a dotted line in FIG. 3.Alternatively, connection to the external computer 30 could be achievedvia a network 40.

FIG. 15 shows an example of one way of transferring data from the mobiledevice to a computer module in an external computer directly via acable. It is envisages that large external servers could be used insteadof personal computers and that these could be contacted via a network 40such as a LAN or WAN.

FIG. 16 shows an example of an output of extracted measurements whichincludes a curl category, median curvature, mean distance index,diameter category and median diameter. Curl category and diametercategory are assigned based on the actual measured values. Examples ofthe curl category and the way in which they are assigned is shown inFIG. 25. These are chosen to provide an easy to reference numericalvalue for curl type. Examples of diameter category are shown in FIG. 27.

Extracted results can be stored in the external computer for futurereference as shown in FIG. 17.

FIG. 18 shows an example of switches of hair grouped according to curl.The relationship between these bought switches and the resultingassigned curl category is shown in FIG. 25. FIG. 19 shows three hairsfrom each group of FIG. 18 placed on the reference card of FIG. 1;

FIG. 20 provides an overview of hair curl descriptions. Averagecurvature, curvature distribution, sign of curvature and maximumcurvature can all be understood in relation to FIG. 21A and 21B.

Curl index (and also mean distance index) can be understood in relationto FIGS. 22 and 23. These drawings also indirectly demonstrate theadvantages of acquiring two images, from a first and second distance.The image on the right of each of FIG. 22 and

FIG. 23 includes a dotted circle delineating a central area. In thisdotted circle, it may appear that there are two separate hairs. However,if another image which is more zoomed-out is also used, such as that onthe left of each of FIGS. 22 and 23, then it is clear that this is infact one curly hair crossing itself numerous times. So, with thisknowledge (i.e. from the first image), it is then possible ensure thatsuperfluous diameter measurements are not made.

FIG. 24 shows curl measurement results for each of the hair types shownin FIGS. 18 and 19 where measured distance index is plotted againstmeasured curvature. FIG. 25 shows the same curl measurements of FIG. 23with curl categories 1 to 10 applied, 1 being the straightest and 10being the curliest.

FIG. 26 shows an example of diameter measurement results for a sample ofhairs of different thicknesses. The algorithm measured diameters areplotted against the actual measurements (LSM mean diameter). It wasfound that the type of mobile device used affected the accuracy of theresults. Results taken using a tablet showed a good correlation, butmapping between measured diameters is not as close to 1:1 as for amobile phone. This is likely to be due to the camera quality, and mayrequire device specific correction factors. Diameter measurements aretherefore highly dependent on device camera performance.

FIG. 27 gives an example of diameter ranges for characterisation.

While the invention has been described in conjunction with the exemplaryembodiments described above, many equivalent modifications andvariations will be apparent to those skilled in the art when given thisdisclosure. Accordingly, the exemplary embodiments of the invention setforth above are considered to be illustrative and not limiting. Variouschanges to the described embodiments may be made without departing fromthe spirit and scope of the invention.

All references referred to above are hereby incorporated by reference.

1. A method of measuring indications of hair type of a user, the methodcomprising the steps of: providing a mobile device; providing areference card, the reference card including one or more referencemarkers; providing one or more of the user's hairs at the surface of thereference card; acquiring one or more images of said one or more hairsusing the mobile device; identifying the path of each of the one or morehairs along the reference card; and calculating the curve of each hairfrom its identified path.
 2. The method of claim 1, wherein the step ofacquiring the one or more images includes the steps of: providing a realtime view of the input of a digital imaging device on the mobile device;providing an overlay on top of the real time view, the overlay havingthe same shape as one or more of the one or more reference markers. 3.The method of claim 1, wherein the step of acquiring an image of saidone or more of the user's hairs includes: obtaining a first image of theone or more hairs on the reference card, the first image being taken ata first distance from the reference card; and obtaining a second imageof the same one or more hairs on the reference card, the second imagebeing taken at a second distance from the reference card; wherein thesecond distance is less than the first distance.
 4. The method of claim3, wherein the reference card comprises: a first reference marker foraligning the mobile decide with the reference card when obtaining thefirst image; and a second reference marker for aligning the mobiledevice with the reference card when obtaining the second image.
 5. Themethod of claim 1, wherein at least three of the user's hairs areprovided at the surface of the reference card.
 6. The method of claim 1,wherein the step of identifying the path of each of the one or morehairs along the reference card comprises: applying an edge detectionalgorithm to the image; identifying the centre line of the one or morehairs at a plurality of points along the hairs; and applying a fit tothe plurality of points;
 7. The method of claim 6, wherein the step ofapplying a fit to the plurality of points includes the step of applyinga polynomial fit.
 8. The method of claim 1, further comprising the stepof extracting the diameter of the one or more hairs from the one or moreacquired images.
 9. A system for measuring the hair type of a user, thesystem comprising: a mobile device; and a reference card, the referencecard including one or more reference markers; wherein the mobile deviceis configured to: acquire one or more images, each of the one or moreimages being an image of one or more hairs of the user; identify thepath of each of the one or more hairs along the reference card; andcalculate the curve of each hair from its identified path.
 10. Thesystem of claim 9, wherein the reference card comprises: a firstreference marker for aligning the mobile decide with the reference cardwhen obtaining a first image using the mobile device; and a secondreference marker for aligning the mobile device with the reference cardwhen obtaining the second image using the mobile device.
 11. The systemof claim 10, wherein the second reference marker is located within thefirst reference marker.
 12. The system of claim 9, wherein the mobiledevice is configured to present the image taken by the digital imagingdevice to the user for quality verification before processing the image.13. (canceled)
 14. A computer program for measuring indications of hairtype of a user, the computer program configured to carry out the stepsof: acquiring one or more images of one or more hairs on a referencecard using a mobile device; identifying the path of each of the one ormore hairs along the reference card; and calculating the curve of eachhair from its identified path.
 15. The computer program of claim 14,wherein the step of acquiring the one or more images includes the stepsof: providing a real time view of the input of a digital imaging deviceon the mobile device; providing an overlay on top of the real time view,the overlay having the same shape as one or more of the one or morereference markers.